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THREE-DIMENSIONAL MODELING 
FROM ARBITRARY THREE-DIMENSIONAL CURVES 



CROSS-REFERENCES TO RELATED APPLICATIONS 

This application claims priority of US provisional patent application serial number 
60/477,933 filed June 13, 2003. 

BACKGROUND OF THE INVENTION 

Field of the Invention 

This invention relates to the field of three-dimensional imaging systems, and more 
particularly to a method and apparatus for creating three-dimensional models of objects 
from sets of arbitrary three-dimensional curves obtained from a target surface. 

Description of Prior Art 

Three-dimensional digitization and modeling systems are commonly used in many 
industries and their applications are numerous. A few examples of such applications are: 
inspection and measurement of shape conformity in industrial production systems, 
digitization of clay models for industrial design and styling applications, reverse 
engineering of existing parts, interactive visualization of objects in multimedia 
applications, and three-dimensional documentation of artwork and artefacts. 

The shape of an object is digitized using a ranging sensor that measures the distance 
between the sensor and a set of points on the surface. From these measurements, three- 
dimensional coordinates of points on the target surface are obtained in the sensor 
reference frame. From a given point of view, the ranging sensor can only acquire distance 
measurements on the visible portion of the surface. To digitize the whole object, the 
sensor must therefore be moved to a plurality of viewpoints in order to acquire sets of 
range measurements that cover the entire surface. A model of the object is built by 
merging and registering the sets of range measurements in a common reference frame. 
Creating the three-dimensional model of an object is a multistep process usually 
comprising: (a) acquiring sets of range measurements from a plurality of viewpoints, (b) 
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roughly aligning the sets of range measurements in a common reference frame, (c) 
refining the registration of the sets of range measurements in the common reference 
frame, (d) merging the range measurements in a unique non-redundant model, and (e) 
building a model of the surface using geometric primitives such as triangles. 

5 Three types of ranging sensors are commonly used to digitize the surface of an object: (a) 
cloud of points sensors which provide unorganized sets of range measurements, (b) range 
image sensors which provide sets of measurements organized into regular matrices, and 
(c) curve sensors which provide range measurements organized into sets of three- 
dimensional curves, e.g. profiles. Cloud of points and range image sensors offer the 
10 advantage of providing a high density of measurements from a single viewpoint. They are 
however bulky and cumbersome to move from one viewpoint to the next. Due to their 
greater simplicity and robustness, curve sensors are usually favored for digitizing the 
entire shape of objects. These sensors can be easily moved from one viewpoint to the 
next, by either a mechanical system or simply an operator holding the sensor in hand. 

15 Numerous examples of ranging sensor devices are described in the art. "Portable digital 
3-d imaging system for remote sites" by J.-A. Beraldin et ah, published in proceedings of 
the 1998 IEEE International Symposium on Circuit and Systems, Monterey, CA, USA: 
326-333. May 31-June 3, 1998, describes a compact laser stripe range sensor that can be 
mounted, for instance on a translational or rotational stage. "A Self-Referenced Hand- 

20 Held Range Sensor" by P. Hebert, published in Proceedings of the IEEE International 
Conference on Recent Advances in 3-D Digital Imaging and Modeling, Quebec, pp. 5-12, 
May 2001, presents a hand-held range sensor gathering a set of curves issued from a 
crosshair laser projection pattern. In "A Review of 20 Years of Ranges Sensor 
Development" by F. Blais, published in, Videometrics VII, in Proceedings of SPDB-IS&T 

25 Electronic Imaging , SPIE Volume 5013 (2003), pp 62-76, NRC 44965, several range 
sensors are described, among these a range scanner collecting both 3-D measurements 
and color (R,G,B) properties is further described in "Digital 3-D Imaging, theory and 
applications" by M. Rioux, published in Proceedings of Videometrics m, SPIE vol. 2350, 
pp.2-15, 1994. 

30 Using curve sensors with current three-dimensional modeling systems requires that the 
sensor be moved in an organized and regular motion perpendicular to the direction of the 
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measured profile. This, to simulate a range image that can be processed in the same way 
the output of a range image sensor would be. Since the range of motion is restricted to a 
single axis, the operation must be repeated multiple times to simulate the acquisition of 
range images from multiple viewpoints. In addition to imposing a regular unidirectional 
5 motion of the sensor, the main drawback with current systems that process data in mis 
manner is that the alignment error of each curve cannot be individually corrected. As the 
set of curves acquired in a given motion sequence is processed as a rigid range image, 
only the global alignment error of the entire set of curves can be corrected by the 
modeling system, thus limiting the accuracy of the reconstructed surface. 

10 No method has been developed in the art for creating a surface model from arbitrary 
three-dimensional curves. U.S. Pat. No. 5,946,645 issued Aug. 31, 1999 to Marc Rioux 
and Patrick Hebert describes a method for individually refining the alignment of arbitrary 
three-dimensional profiles. The method claimed relies on measuring the distance between 
each profile and every other profile in the set. Because of this combinatorial complexity, 

15 the method cannot currently be used in practice with the high number of curves typically 
needed to model an entire object. Furthermore, mis patent does not describe any means 
for creating a model of the surfece from the three-dimensional curves once their 
alignment has been refined. The use of this curve registration method on data acquired 
from a hand-held sensor is described in 'Toward a hand-held laser range scanner: 

20 integrating observation-based motion compensation" by P. Hebert et al., published in 
Proceedings of SPiE, volume 3313, pages 2-13, January 1998. 

A volumetric method for reconstructing surface models from unorganized clouds of 
three-dimensional points is described in "Surface Reconstruction from Unorganized 
Points" by H. Hoppe et al., published in SIGGRAPH '92 Proceedings, volume 26, pages 

25 71-78, July 1992. The volumetric representation used by this method only contains the 
distance to the surface, which is sufficient to reconstruct the surface but does not contain 
enough information to refine alignment errors of data sets. This method is general and can 
be used to create a surface model for a set of arbitrary three-dimensional curves by 
converting it into an unorganized cloud of points. This process is difficult to implement in 

30 practice because it requires that the entire target surface be measured with a high and 
relatively uniform density. Doing so also eliminates the rigid arrangement of each curve 
in the set This loss of information negatively impacts the quality and accuracy of the 
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reconstructed surface because it becomes impossible to correct the alignment of curves. 
No method for refining the alignment of the three-dimensional data is described. A 
specialization of this surface reconstruction method for merging range images is 
described in, "A Volumetric Method for Building Complex Models from Range Images" 
by B. Curless et al., published in SIGGRAPH '96, Proceedings, pages 303-312, 1996. 
This is a volumetric method that reconstructs the surface incrementally by adding range 
images to the model. The method exploits the structure of the images and cannot be used 
with arbitrary three-dimensional curves. As with the more general approach described by 
Hoppe, no method for refining the alignment of the three-dimensional data is described. 

A method for reconstructing a surface from data acquired with a hand-held curve sensor 
is described in "Geometric Fusion for a Hand-Held 3D Sensor" by A. Hilton et al., 
published in Machine Vsion and Applications, 12:44-51, 2000. This method requires that 
the sensor is moved in a regular and organized motion in such a way to provide parallel 
curves that are merged into range images. Instead of directly using the three-dimensional 
curves to build the surface, this method must combine the curves into simulated range 
images. The main drawback of the method is that it poses severe constraints on the range 
of motion of the sensor. Also, since the curves are merged into images, the alignment 
error of each curve cannot be individually corrected. 

"Registration and Integration of Multiple Range Images by Matching Signed Distance 
Fields for Object Shape Modeling" by T. Masuda published in Computer Vision and 
Image Understanding 87, pages 51-65, 2002 describes a volumetric method for surface 
reconstruction and alignment refinement based on signed distance fields. This method is 
however specific to range images and cannot be generalized to arbitrary three- 
dimensional curves because signed distance fields cannot be constructed from single 
curves. 

There therefore exists a need for new methods for creating three-dimensional models of 
objects from sets of arbitrary three-dimensional entities obtained from a target surface and 
elimmating the need for moving the sensor in organized and regular motions to simulate 
the creation of range images. 
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SUMMARY OF THE INVENTION 

It is an object of the present invention to overcome the problems mentioned previously. 

In a preferred embodiment, the present invention provides a method for ascertaining the 
three-dimensional shape of a target surface using a hand-held ranging sensor to obtain a 
5 plurality of arbitrary overlapping profiles. 

The present invention relates to a method for creating three-dimensional models of 
objects from sets of arbitrary three-dimensional entities obtained from target surfaces. It 
also provides an efficient method for individually refining the alignment of curves to 
improve the accuracy of the surface model with a linear complexity with respect to the 

10 number of curves. The principle behind the invention is that a set of three-dimensional 
entities, at their approximate positions, creates a field from which the surface can be 
extracted. The field is constructed in a manner such that the three-dimensional entities are 
attracted toward the extracted surface. This attraction is used to accurately register each 
three-dimensional entity with respect to extracted surface. Through iterations, both the 

.15 field and the entity positions are refined t 

According to one broad aspect of the present invention, there is provided a method for 
reconstructing surfaces from a single or a plurality of arbitrary three-dimensional entities 
obtained from a target surface. The method comprises: obtaining a set of at least one 
three-dimensional entity, each entity being a set of three-dimensional points, each three- 

20 dimensional point containing at least the three-dimensional coordinate of the point on the 
target surface; constructing a volumetric implicit representation of the target surface in 
the form of a vector field, each vector in the vector field containing at least the distance to 
the reconstructed surface and the direction toward the reconstructed surface; placing each 
three-dimensionalentity in the vector field and updating the vector field accordingly; and 

25 Reconstructing the target surface from the information contained in the vector field. 



Preferably, at least one of the three-dimensional entities is an organized set of three- 
dimensional points that are part of one or a plurality of three-dimensional curves. 
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According to another broad aspect of the present invention, there is provided a method for 
refining the alignment of arbitrary three-dimensional entities obtained from a target 
surface. The method comprises: 

(a) obtaining a set of at least two three-dimensional entities, each entity being a set 
5 of three-dimensional points, each three-dimensional point containing at least the 

three-dimensional coordinate of the point on the target surface; 

(b) constructing a volumetric implicit representation of the target surface in the 
form of a vector field, each vector in the vector field containing at least the 
distance to the reconstructed surface and the direction toward the reconstructed 

10 surface; 

(c) placing at least two three-dimensional entities in the vector field and updating 
the vector field accordingly; 

(d) selecting at least one three-dimensional entity placed in the vector field and 
obtaining a subset of three-dimensional points on each of the selected three- 

15 dimensional entities, three-dimensional points in these subsets being called control 

points; 

(e) for each control point in each selected three-dimensional entity, computing a 
contribution to a cost function, this contribution being a function of at least the 
vector field and the three-dimensional coordinate of the control point; 

20 (f) for each selected three-dimensional entity, computing a new position that 

optimizes its corresponding cost function; and 

(g) placing each selected three-dimensional entity in the vector field at its newly 
computed position and updating the vector field accordingly. 

According to a further aspect of the present invention, the steps (c) and (d) of the previous 
25 method can be modified for: 

(c) placing at least one three-dimensional entity in the vector field and updating 
the vector field accordingly; 

(d) selecting at least one of the three-dimensional entities not yet placed ia the 
vector field, placing the selected three-dimensional entities in the vector field 
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without updating the field and obtaining a subset of three-dimensional points on 
each of the selected entities, three-dimensional points in these subsets being called 
control points. 

According to still another broad aspect of the present invention, there is provided a 
5 method for ascertaining the three-dimensional shape of a target surface. The method 
comprises using a ranging sensor to produce a set of three-dimensional entities, a method 
according to present invention to reconstruct the target surface, and a method according to 
the present invention to improve the quality and accuracy of the reconstructed surface. 

According to another broad aspect of the present invention, there is provided a system for 
10 reconstructing surfaces from a single or a plurality of arbitrary three-dimensional entities 
obtained from a target surface comprising: a three-dimensional entity provider for 
obtaining a set of at least one three-dimensional entity, each entity being a set of three- 
dimensional points, each three-dimensional point containing at least the three- 
dimensional coordinates of the point on the target surface; an implicit representation 
15 constructor for constructing a volumetric implicit representation of the target surface in 
the form of a vector field, each vector in the vector field containing at least the distance to 
the reconstructed surface and the direction toward the reconstructed surface; for placing 
each three-dimensional entity in the vector field; and for updating the vector field 
accordingly; and a target surface reconstructor for reconstructing the target surface from 
20 the information contained in the vector field. 

According to a further broad aspect of the present invention, there is provided a system 
for refining an alignment of arbitrary three-dimensional entities obtained from a target 
surface, comprising: a three-dimensional entity provider for obtaining a set of at least two 
three-dimensional entity, each entity being a set of three-dimensional points, each three- 

25 dimensional point containing at least the three-dimensional coordinates of the point on the 
target surface; an implicit representation constructor for constructing a volumetric 
implicit representation of the target surface in the form of a vector field, each vector in 
the vector field containing at least the distance to the reconstructed surface and the 
direction toward the reconstructed surface; for placing at least two tlnee-dimensional 

30 entity in the vector field; and for updating the vector field accordingly; and a control 
point selector for selecting at least one three-dimensional entity placed in the vector field 
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and obtaining a subset of three-dimensional points on each of the selected three- 
dimensional entities, three-dimensional points in these subsets being called control points; 
a cost function calculator for computing, for each control point in each selected three- 
dimensional entity, a contribution to a cost function, the contribution being a function of 
5 at least the vector field and the three-dimensional coordinate of the control point; a new 
position calculator for computing, for each selected three-dimensional entity, a new 
position that optimizes its corresponding cost function; wherein the implicit 
representation constructor places each selected three-dimensional entity in the vector field 
at its newly computed position and updates the vector field accordingly. 

10 According to still a further broad aspect of the present invention, there is provided a 
system for refining an alignment of arbitrary three-dimensional entities obtained from a 
target surface, comprising: a three-dimensional entity provider for obtaining a set of at 
least two three-dimensional entity, each entity being a set of three-dimensional points, 
each three-dimensional point containing at least the three-dimensional coordinates of the 

15 point on the target surface; an implicit representation constructor for constructing a 
volumetric implicit representation of the target surface in the form of a vector field, each 
vector in the vector field containing at least the distance to the reconstructed surface and 
the direction toward the reconstructed surface; for placing at least one three-dimensional 
entity in the vector field; and for updating the vector field accordingly; and a control 

20 point selector for selecting at least one of the three-dimensional entities not yet placed in 
the vector field, wherein the implicit representation constructor places the selected three- 
dimensional entities in the vector field without updating the field, and obtaining a subset 
of three-dimensional points on each of the selected three-dimensional entities, three- 
dimensional points in these subsets being called control points; a cost function calculator 

25 for computing, for each control point in each selected three-dimensional entity, a 
contribution to a cost function, the contribution being a function of at least the vector field 
and the three-dimensional coordinate of the control point; a new position calculator for 
computing, for each selected three-dimensional entity, a new position that optimizes its 
corresponding cost function; wherein the implicit representation constructor places each 

30 selected three-dimensional entity in the vector field at its newly computed position and 
updates the vector field accordingly. 
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In a preferred embodiment, the invention is used to process three dimensional curves 
obtained from a hand held sensor such as that described in "A Self-Referenced Hand- 
Held Range Sensor" by P. Hebert, published in Proceedings of the IEEE International 
Conference on Recent Advances in 3-D Digital Imaging and Modeling, Quebec, pp. 5-12, 
5 May 2001. 



BRIEF DESCRIPTION OF THE DRAWINGS 

These and other features, aspects and advantages of the present invention will become 
better understood with regard to the following description and accompanying drawings 
10 wherein: 

Fig. 1 represents a flowchart describing the method for surface reconstruction from 
arbitrary three-dimensional curves followed by a registration of three-dimensional curves 
to the reconstructed surface; 

Fig. 2 represents the initialization step; 

15 Fig. 3 represents the preprocessing step; 

Fig. 4 represents the model reconstruction step; 

Fig. 5 represents the integration of a single curve into the volumetric structure as a step 
for model reconstruction; 

Fig. 6 represents the determination of the distance d between a voxel and a line segment 
20 in the integration of a single curve step; 

Fig. 7 represents the curve registration step; 

Fig. 8 represents the registration procedure for a single curve; 

Fig. 9 represents the extrapolation of the matching point m y for the control point c y in the 
registration procedure for a single curve; 
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Fig. 10 represents a flowchart where each curve is registered prior to its integration to the 
volumetric structure; 

Fig. 1 lrepresents a flowchart of the main steps of one embodiment of the invention; 

Fig. 12 represents a flowchart of the main steps of a second embodiment of the invention; 

5 Fig. 13 represents a flowchart of the main steps of a third embodiment of the invention; 
and 

Fig. 14 represents a block diagram of the main components of the present invention. 

DESCRIPTION OF PREFERRED EMBODIMENT 

10 The present invention now will be described more fully hereinafter. However, this 
invention may be embodied in many different forms and should not be construed as 
limited to the embodiments set forth herein; rather, these embodiments are provided so 
that this disclosure will be thorough and complete, and will fully convey fhe scope of the 
invention to those skilled in the art. 

15 Figure 1 shows a flowchart describing the functioning of the method for surface 
reconstruction from arbitrary 3D curves as well as for registration of 3D curves to the 
reconstructed surface. 

INITIALISATION 

Initialization step 100, illustrated in Figure 2, requires two user-supplied parameters: 
20 bounding box of the object to be reconstructed and the resolution of the volumetric lattice 
of points (volumetric structure). 

The bounding box is defined as coordinates b min = [x^y^z^f and 

max t^max » .V max > ^max 

] of two opposite comers of the box that completely contains the 
object, such that x mitt < x mm , y^ < y^ and z min < z max . 
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Resolution A of the volumetric structure is defined as the distance between two 
neighbouring lattice points. 

Volumetric lattice V of points is defined 202 as a set Voi points (voxels) 

F = tW;.z*] r |^ = * min +i'A,v, = Vmin +jA,z k =z min +k&}, 
i = 0,...,N x , j = 0,...,N y ,k = 0,...,N z , where 



AT = 



•*max -^rnin 



, N y = 



y max ^min 

A 



and AT = 



2 max Z min 



Operator f" x] represents smallest integer greater than x . 

A 3x 3 matrix K (covariance matrix) is attached to each lattice point and all elements of 
the matrix K are initialized to zero 203. Also, to each lattice point, a vector g (distance 
10 vector) is attached as well as a weighting factor® both initialized to zero. This 
volumetric structure represents implicitly the reconstructed surface. 

INPUT DATA 

Input 101 to the reconstruction and registration algorithm is a set C = {Ci,C 2 ,...,C w } of 
3D curves measured on the surface of an object. A curve C, is defined as a set of 3D 
1 5 points C, ={p 13 p 2 ,...,p^}, p A = [x t y k z k f such that the real surface curve can be 
approximated as a set of line segments {piP 2 >P 2 P3>--->Pa/,-iPa/ ( 1- 
PREPROCESSING 

The only preprocessing step 102, Figure 3, is the estimation of tangents at each point of 
all curves 304. At a point p k of curve C, = jp,^,...^} the tangent t.can be 
20 estimated as: 



t =1 P*~P*-i , P*+i~P* 
* 2|>.-P«| IP^PJ. 

Optionally, the tangents can be filtered by averaging them with tangents at neighbouring 
points: 
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After the preprocessing step, a set of tangents T t = {t„t 2 t u J is assigned to each 

curve C, 307. To distinguish the tangents belonging to different curves, the box 304 
specifies t tt as a single tangent, where i is the curve index. 

5 MODEL RECONSTRUCTION 

Model reconstruction 103, Figure 4, consists of the integration of each curve 
independently into the volumetric structure 404. The order in which the curves are 
integrated is not important. The final result is the same regardless the order of the curves. 



10 



INTEGRATION OF A SINGLE CURVE INTO THE VOLUMETRIC STRUCTURE 



The integration of a single curve into the volumetric structure 404 is depicted as a 
flowchart diagram in Figure 5. For the purpose of integration of curves into the 
volumetric structure, a curve C, = {p„p 2 ,...,p„ ( } is considered as a set of line segments 
{PiP2>P2P3»---»Pji / ,_iP W( }. A curve is always integrated one line segment at the time 504. 
In the following, the computation is explained for a single line segment p k p k+1 . The same 
1 5 procedure is repeated for all other line-segments contained in the curve 514. 

The integration procedure consists in updating the covariance matrices and the distance 
vectors in the neighbourhood of the line segment. The size of the neighbourhood s is a 
user-specified parameter and cannot be smaller then the -V3A , where A is the resolution 
of the volumetric structure, as defined above. 

20 The first step of integration 404 is to compute the bounding box B for the line segment. 
The bounding box is defined as coordinates b min = K in ,^ min ,z mto f and 
b max = IXnax'-VVax > Z m m Y of two opposite comers of a rectangular box such that 

*mte < X oiax » 

^min <y mm and' 



25 



^min ^ ^max * 
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where 

*mi» = mine** - e, x M -e), x mttl = max(x k + s, x k+1 + e) 
^min = min(y t - e,y M - e) , y mm = max(y k + e, y k+l + e) 
z min = min(z t - e, z M -s), z max = max(z A + s, z kU + e) 
5 Let L be the set of all lattice points (voxels) within the bounding box B 
L = e V | Vj g B,j = 1,.. , where Vis the set of lattice points 202. For each voxel 
v y e L it is first verified that the voxel v y is in-between two planes II, and Il 2 defined 
as planes that contain points p fc and p t+1 and whose normals are equal to tangents t k and 
*jt+i respectively 510. If not, the next voxel is selected 507. 

10 To compute the distance, a line segment p t p t+1 is parameterized as: 

K") =P* +u(p k+1 -p t ),0 < u <, 1. 
The tangents at the two end points are then interpolated as: 

t(«) = t 4 +«(t Jl+l -t t ),0£ M ^l. 
The normal n(«) at each point of a line segment is defined as the vector being contained 
15 in the plane p ft p t+1 v y and perpendicular to the tangent t(w) . The normal n(u) and is also 
interpolated over the line segment in the same manner: 

n(u) = n»+«(n iw -n i ) 1 0^«<l. 
Normals n k and n i+ , are unit vectors perpendicular to tangents t k and t M respectively 
and contained in plane PtP^, v y . 

20 The distance d between voxel v, and the line segment p t p A+I is defined as the distance 
between v, and the point p c whose normal n c passes through v, . This is illustrated in 
Figure 6. The distance is computed as the distance for which the area of the triangle 
<h*l*+i v j is zero, i.e. that the cross-product of q k - v y and q A+ , - v y is zero, where 

q t =<m i+ p A , 

25 = dn k» + P*+i- 
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The distance d is obtained from the equation 

a + db + rf 2 c = 0, 

where 

b = n, +1 x (p k - v,) + n, x (p M - v, ) , 
a-f db + d 2 c = 0. 

Preceding equation is a system of three quadric equations with a single unknown d. Any 
of these equations can be used to compute d after making sure that the chosen parameters 
10 a,b, c do not vanish altogether. The chosen equation can have up to two real roots. If the 
number of roots is zero, the voxel v, is rejected and next voxel is selected 507. If there 
are two real roots, the root of interest is the one with the smallest positive value. 

Using the distance computed above, parameter u c is computed as: 



20 



= (!<**- y 

15 Next, it is verified whether the distance d is smaller then the maximal allowed distance e . 
If not, next voxel is selected 507. 

Finally, the covariance matrix K } at voxel \j is updated as follows: 
Vector gj is updated as: 



g y <-g,+e °- 5 *(v y -p c ) 
Weighting value <Dj is updated as: 



The whole procedure is repeated for the remaining voxels 514 as well as for the 
remaining line segments 515. 
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The resulting volumetric structure contains an implicit representation of the reconstructed 
surface (reconstructed model). The covariance matrices K t and distance vectors g, 
represent implicitly a vector field whose value is the direction and the distance toward the 
closest point on the reconstructed model. 

5 CURVE REGISTRATION 

Each curve is registered independently 604 to the reconstructed model that is implicitly 
represented in the volumetric structure. Flowchart in Figure. 7 illustrates the overall 
procedure. 

REGISTRATION OF A SINGLE CURVE 

10 Figure 8 shows registration procedure for a single curve C f = {p,,p 2 ,...,p^ f }. First a 
subset W t of points in C k is chosen 702 as a set of control points 

W t sC, =(c„c 2 ,...,c e J 

(The set W t can be equal to C, ) 

15 The following procedure is repeated for all control points c, . First the closest voxel v c to 
the control point c, is found. Then the direction f from point c, toward the 
reconstructed model is computed using the covariance matrix K c at voxel v c . The 
direction f is the eigenvector associated with the smallest eigenvalue of matrix K c lo) c9 
where <d c is the weighting factor at voxel v c . The distance e to the reconstructed model 

20 is obtained as (f ,g c )/a> c . The matching point m,. for the control point c y is extrapolated 
as 

m y =c y +f[e-Kf } (v c -c ; )>J 
which is illustrated in figure 9. The value ef is a vector that indicates the direction and 
the distance toward the closest point on the reconstructed model from the voxel v c . 
25 Once the matching point is obtained for all control points, a rigid transformation G> that 
best aligns two sets of points {c l5 c 2 ,...,c y } and {m l ,m 2 ,...,m / } is computed. This is 
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10 



accomplished using standard techniques, for example quaternions. The transformation 
matrix <3> is then applied to all curve points: 

P/ <-3>P,, / = 0,...,Af,. 
An example of a cost function that can be used to compute the rigid transformation <I> 
is a sum of squared distances between corresponding points, i.e. 

The cost function can be modified to exploit surface proprieties to improve the quality of 
registration. For example, the following cost function utilises color T = [iJ G Bf 
associated with each point: 



j 

where Z k is a weighting factor dependent on similarity of colors: 

1 if colors T ck and are si 
otherwise 

andr^ are colors associated with points and respectively. 



similar^ 



Prior to computing matching points it has to be verified whether the eigenvalues 

15 \ < ^ < A3 of matrix K c lco c satisfy the following relation ^ > 0.05 and \ < 0.5^ . 

These values are provided as indications. They mean that the local distribution of tangents 

is nearly planar. If not, both control point c ; and matching point m, are rejected. When 

more than a given number of points are rejected (for example 70%) then the whole curve 
is rejected. 

20 If the average displacement of curve points after the transformation has been applied is 
larger then a threshold (for example 1% of the grid resolution A) the whole procedure is 
repeated 710. 

SURFACE EXTRACTION 

The surface extraction proceeds by converting the implicit surface representation in the 
25 volumetric structure into scalar distance field. This is performed by computing the 
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direction f,and distance e from each voxel v } toward the reconstructed model if the 
co>0 at voxel v y . The direction f is eigenvector associated with the smallest 
eigenvalue of matrix K } lm }> where K } is covariance matrix and a> is the weighting 
factor at voxel v, . The distance e to the reconstructed model is obtained as {t Jt gj}/<Oj . 
5 Distance e at each voxel represents a scalar field. A Marching Cubes algorithm, for 
instance, is applied to mis scalar field in order to obtain a triangulated surface 
(represented as a set of triangles). 

ALTERNATIVE EMBODIMENTS AND APPLICATIONS 
1 0 Reconstruction of additional surface properties 

Presented method can be used to reconstruct not only model geometry, but also different 
surface properties including but not limited to color, intensity, texture and temperature. 
To allow reconstruction of additional object properties the following modifications are 
15 required: 

Each curve is assumed to have a surface property assigned to each curve point, i.e. for a 
curve C, = |pi,P 2 ,...,p M } alproperty h, is assigned to each point p,. 

During the initialization step 100 a single or multiple zero valued properties H are 
attached to each lattice point. Surface properties can be either scalar or vector valued. 
20 During the integration of curves 404, property value H y at voxel v y is updated as: 

H ; *-H y +© y (li l +« B (h w -h J )). 
During the extraction of the triangulated surface (by Marching Cubes), property value 
interpolated between two voxels closest to each triangle vertex. 

Alternative configuration: Incremental registration 

25 The two basic building blocks of the method, curve integration 404 and curve registration 
604 can be rearranged into a different configuration to accomplish incremental 
registration where each curve is registered prior to its integration. The flowchart in figure 
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10 illustrates the principle of functioning. All building blocks remain the same as for the 
configuration in Figure 1. 

Modeling from different surface entities 

The method presented above accepts as input a set of non-parallel 3D curves which 
5 generates organized set of three-dimensional points. However, the same method can be 
used to build and update the volumetric structure from various arbitrary three- 
dimensional entities such as partial surfaces (range images), parallel curves, unorganized 
clouds of points or any combination of these. Moreover, the three-dimensional points can 
be measured in a single plane and the coordinate system can be reduced to a two- 
10 dimensional system. Required modification for each one of these cases is discussed 
below. 

Modeling from Partial Surfaces (Range images) 

Range images are matrices of points measured on the surface of an object in such a way 
as to allow an approximation of the real surface by a set of connected triangles. Such a 

15 surface Sis defined as a set of O triangles S = {A X ,A 2 ,... 9 A 0 }, where each triangle A, is 
defined by three vertices p n , p, 2 and p i3 . It is also assumed that a normal n {J is 
estimated at each vertex, which can be easily accomplished using standard techniques. 
The surface is integrated into the volumetric structure one triangle at the time. As for the 
line segments, a bounding box is estimated for the triangle and for each voxel v^ within 

20 bounding box the closest point p c on the triangle is found. The elements attached at the 
voxel are then updated as follows: 

Kj+-Kj+t x t 2 T 

Vector gj is updated as: 

25 gj *-g y +e °' 5A (v y -p c ) 

Weighting value o)j is updated as. 
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d 2 

where d = ||v y -p c ||, and t x and t 2 are two vectors, perpendicular to each other as well 
as to the normal at p c . 

Surface is registered to the reconstructed model by choosing a subset of (or all) vertices 
5 of the input surface. 

Modeling from parallel curves 

In order to allow the reconstruction from parallel curves, the elements attached at each 
voxel Vj are updated as follows: 

10 

Kj ^ Kj +tl t/ + (y / - Pc )( Vy - Pc f 

Vector gj is updated as: 

S f <r-gj+e °- 5A (v y -p c ) 
Weighting value (Oj is updated as: 

d 1 

15 co j <r-a>j +e °- 5A . 

Prior to the registration, or surface extraction the covariance matrix K c is changed to 

£ c <-K-Sjgj T 

Modeling for unorganized sets of points 
20 An unorganized P set of points is simply a set of 3D Points P = {p t ,p 2 , . . .,p^ }. Model 
reconstruction is performed one point at a time. For a point p,, a set of voxels contained 
in the sphere centered at p, is found first. Radius of sphere is e . Then the elements 
attached at each voxel v y are updated as follows: 

Kj <- Kj+(y j -P,Xy j -P.f 
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Vector gj is updated as: 
Weighting value <o s is updated as. 

a>j <r-a>j+e 0 5A 

5 where d = ||v y -p,||. 

As shown in Fig. 11, one embodiment of the preferred invention, namely a method for 
reconstructing surfaces from a single or a plurality of arbitrary three-dimensional entities 
obtained from a target surface, comprises the following steps: obtaining a set of at least 
one three-dimensional entity 1100, each entity being a set of three-dimensional points, 

10 each three-dimensional point containing at least the three-dimensional coordinates of the 
point on the target surface; constructing a volumetric implicit representation of the target 
surface in the form of a vector field 1101, each vector in the vector field containing at 
least the distance to the reconstructed surface and the direction toward the reconstructed 
surface; placing each three-dimensional entity in the vector field and updating the vector 

15 field accordingly 1102; and reconstructing the target surface from the information 
contained in the vector field 1 103. 

In a method for ascertaining the three-dimensional shape of a target surface, an additional 
step of using a ranging sensor to produce a set of three-dimensional entities 1104 is 
carried out 

20 As shown in Fig. 12, another embodiment of the preferred invention, namely a method 
for refining the alignment of arbitrary three-dimensional entities obtained from a target 
surface, comprises the following steps : obtaining a set of at least two three-dimensional 
entities 1200, each entity being a set of three-dimensional points, each three-dimensional 
point containing at least the three-dimensional coordinate of the point on the target 

25 surface; constructing a volumetric implicit representation of the target surface in the form 
of a vector field 1201, each vector in the vector field containing at least the distance to the 
reconstructed surface and the direction toward the reconstructed surface; placing at least 
two three-dimensional entities in the vector field and updating the vector field 
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accordingly 1202; selecting at least one three-dimensional entity placed in the vector field 
and obtaining a subset of three-dimensional points on each of the selected three- 
dimensional entities, three-dimensional points in these subsets being called control points 
1203; for each control point in each selected three-dimensional entity, computing a 
contribution to a cost function 1204, the contribution being a function of at least the 
vector field and the three-dimensional coordinate of the control point; for each selected 
three-dimensional entity, computing a new position 1205 that optimizes its corresponding 
cost function; and placing each selected three-dimensional entity in the vector field at its 
newly computed position and updating the vector field accordingly 1206. 



As shown in Fig. 13, another embodiment of the preferred invention, namely a method 
for refining the alignment of arbitrary three-dimensional entities obtained from a target 
surface, comprises the following steps: obtaining a set of at least two three-dimensional 
entities 1300, each entity being a set of three-dimensional points, each three-dimensional 
point containing at least the three-dimensional coordinate of the point On the target 
1 5 surface; constructing a volumetric implicit representation of the target surface in the form 
of a vector field 1301, each vector in the vector field containing at least the distance to the 
reconstructed surface and the direction toward the reconstructed surface; placing at least 
one tliree-dimensional entity in the vector field and updating the vector field accordingly 
1302; selecting at least one of the three-dimensional entities not yet placed in the vector 
20 field 1303, placing the selected three-dimensional entities in the vector field without 
updating the field 1304 and obtaining a subset of three-dimensional points on each of the 
selected three-dimensional entities, three-dimensional points in these subsets being called 
control points 1305; for each control point in each selected three-dimensional entity, 
computing a contribution to a cost function 1306, the contribution being a function of at 
25 least the vector field and the three-dimensional coordinate of the control point; for each 
selected three-dimensional entity, computing a new position 1307 that optimizes its 
corresponding cost function; and placing each selected three-dimensional entity in the 
vector field at its newly computed position and updating the vector field accordingly 
1308. 

30 As shown in Fig. 14, a preferred embodiment of the system for reconstructing surfaces 
from a single or a plurality of arbitrary three-dimensional entities obtained from a target 
surface comprises the following main components: a three-dimensional entity provider 
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1401 for obtaining a set of at least one three-dimensional entity, each entity being a set of 
three-dimensional points, each three-dimensional point containing at least the three- 
dimensional coordinates of the point on the target surface; an implicit representation 
constructor 1402 for constructing a volumetric implicit representation of the target 
5 surface in the form of a vector field, each vector in the vector field containing at least the 
distance to the reconstructed surface and the direction toward the reconstructed surface; 
for placing each three-dimensional entity in the vector field and for updating the vector 
field accordingly, and a target surface reconstructor 1403 for reconstructing the target 
surface from the information contained in the vector field. Preferably, if the system is 
10 used to ascertain the three-dimensional shape of a target surface, a ranging sensor 1400 is 
used to produce the set of three-dimensional entities. 

If the system is used to refine the alignment of arbitrary three-dimensional entities 
obtained from a target surface, the following components are provided: a control point 
selector 1404 for selecting at least one three-dimensional entity placed in the vector field 

15 and obtaining a subset of three-dimensional points on each of the selected three- 
dimensional entities, three-dimensional points in these subsets being called control points; 
a cost function calculator 1405 for computing, for each control point in each selected 
three-dimensional entity, a contribution to a cost function, the contribution being a 
function of at least the vector field and the three-dimensional coordinate of the control 

20 point; for each selected three-dimensional entity, a new position calculator 1406 for 
computing a new position that optimizes its corresponding cost function. The implicit 
representation constructor 1402 is then used to place each selected three-dimensional 
entity in the vector field at its newly computed position and updating the vector field 
accordingly. 

25 If the system is used to refine the alignment of arbitrary three-dimensional entities 
obtained from a target surface, , the following components are provided: a control point 
selector 1404 for selecting at least one of the three-dimensional entities not yet placed in 
the vector field, using the implicit representation constructor 1402 to place the selected 
three-dimensional entities in the vector field without updating the field and obtaining a 

30 subset of three-dimensional points on each of the selected three-dimensional entities, 
three-dimensional points in these subsets being called control points; a cost function 
calculator 1405 for computing, for each control point in each selected three-dimensional 
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entity, a contribution to a cost function, the contribution being a function of at least the 
vector field and the three-dimensional coordinate of the control point; a new position 
calculator 1406 for computing, for each selected three-dimensional entity, a new position 
that optimizes its corresponding cost function; and using the implicit representation 
5 constructor 1402 to place each selected three-dimensional entity in the vector field at its 
newly computed position and updating the vector field accordingly. 

While illustrated in the block diagrams as groups of discrete components communicating 
with each other via distinct data signal connections, it will be understood by those skilled 
in the art that the preferred embodiments are provided by a combination of hardware and 
10 software components, with some components being implemented by a given function or 
operation of a hardware or software system, and many of the data paths illustrated being 
implemented by data communication within a computer application or operating system. 
The structure illustrated is thus provided for efficiency of teaching the present preferred 
embodiment. 

15 It will be understood that numerous modifications thereto will appear to those skilled in 
the art. Accordingly, die above description and accompanying drawings should be taken 
as illustrative of the invention and not in a limiting sense. It will further be understood 
that it is intended to cover any variations, uses, or adaptations of the invention following, 
in general, the principles of the invention and including such departures from the present 

20 disclosure as come within known or customary practice within the art to which the 
invention pertains and as may be applied to the essential features herein before set forth, 
and as follows in the scope of the appended claims. 



